In order to surveil or analyze the state of the crowd, a video imaged by a surveillance camera fixed to a building or the like on the moving route of the crowd is used. For example, Patent Document 1 discloses a technique for analyzing an input surveillance video to calculate the moving direction of the crowd, and controlling a surveillance device according to the calculated moving direction.